{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mNESTCLfvSrY"
      },
      "source": [
        "# Welcom to SSCMA for Google Colab Training Example 🔥\n",
        "\n",
        "<a href=\"https://colab.research.google.com/github/Seeed-Studio/SSCMA/blob/main/notebooks/Google-Colab-SWIFT-YOLO-A1101-Example.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a> **[🚀🚀🚀 One-Click to Deploy to Google Colab 🚀🚀🚀](https://colab.research.google.com/github/Seeed-Studio/SSCMA/blob/main/notebooks/Google-Colab-SWIFT-YOLO-A1101-Example.ipynb)**\n",
        "\n",
        "In this tutorial, we will demonstrate how to use [SSCMA](https://github.com/Seeed-Studio/SSCMA) and Colab to train a SWFIT-YOLO model for meter reading ⏱️.\n",
        "\n",
        "Table of contents:\n",
        "\n",
        "- [Setup SSCMA](#setup-SSCMA)\n",
        "- [Download Custom Meter Datasets](#download-custom-meter-datasets)\n",
        "- [Train SWFIT-YOLO Model](#train-yolov5-tiny-model)\n",
        "- [Optimize SWFIT-YOLO Model and Export](#optimize-yolov5-tiny-model-and-export)\n",
        "- [Deploy to SenseCAP A1101](#deploy-to-sensecap-a1101)\n",
        "- [Thanks for Trying out SSCMA 🎉](#thanks-for-trying-out-SSCMA-🎉)\n",
        "\n",
        "For more details of SWIFT-YOLO model or application scenarios, please refer to [SSCMA Documentation](https://seeed-studio.github.io/SSCMA/tutorials/training/yolov5).\n",
        "\n",
        "**Tips: Since model training consumes a lot of computational resources, we recommend that you run this notebook on [Colab's GPU instances](https://research.google.com/colaboratory/faq.html#gpu-availability).**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PRGi8-2jvxzR"
      },
      "source": [
        "## Setup sscma"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ikzx6C6wrpos"
      },
      "source": [
        "**Step 0:** Clone the SSCMA source code from [SSCMA GitHub repository](https://github.com/Seeed-Studio/SSCMA), and enter the SSCMA project directory."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "F5pMIwXuvPpG"
      },
      "outputs": [],
      "source": [
        "!git clone https://github.com/Seeed-Studio/SSCMA.git  # currently we're using experimental 2.0 version branch\n",
        "%cd /content/SSCMA"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LWAE--J9sAmW"
      },
      "source": [
        "**Step 1:** Install python third-party library"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "EPSObA30OADp"
      },
      "outputs": [],
      "source": [
        "!pip install ."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "v47-pxMe2DP1"
      },
      "source": [
        "## Download Custom Meter Datasets\n",
        "\n",
        "We'll download our dataset from Roboflow. Use the \"**COCO**\" export format. Note that the Ultralytics implementation calls for a YAML file defining where your training and test data is. The Roboflow export also writes this format for us.\n",
        "\n",
        "To get your data into Roboflow, follow the [Getting Started Guide](https://blog.roboflow.ai/getting-started-with-roboflow/).\n",
        "<p align=\"\"><a href=\"https://roboflow.com/?ref=ultralytics\"><img width=\"480\" src=\"https://uploads-ssl.webflow.com/5f6bc60e665f54545a1e52a5/6152a275ad4b4ac20cd2e21a_roboflow-annotate.gif\"/></a></p>Label images lightning fast (including with model-assisted labeling)\n",
        "\n",
        "- open our digital meter dataset [seeed_meter_digit](https://universe.roboflow.com/seeeddatasets/seeed_meter_digit/)\n",
        "- Download Dataset with COCO format\n",
        "- Copy the download code and paste it into the below\n",
        "\n",
        "***More Seeed Studio public datasets are available [here](https://universe.roboflow.com/seeeddatasets).***"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "KzjPv7VQL42Q"
      },
      "outputs": [],
      "source": [
        "!mkdir -p datasets\n",
        "!wget \"https://universe.roboflow.com/ds/GDH5ex194I?key=dPovYB5xI6\" -O datasets/digital_meter.zip\n",
        "!unzip -q datasets/digital_meter.zip -d datasets/digital_meter && rm datasets/digital_meter.zip"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2HMAOOWR2sf6"
      },
      "source": [
        "## Train YOLOv5 Tiny Model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zmWPp53tcLpB"
      },
      "source": [
        "### Examine the YOLOv5 Tiny config file\n",
        "\n",
        "The config we use for train is located at `configs/swift_yolo/swift_yolo_tiny_1xb16_300e_coco.py`.\n",
        "\n",
        "For more config details, please refer to [SSCMA Documentation - Config](https://seeed-studio.github.io/SSCMA/tutorials/config)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "J7TrWv8_wjXY"
      },
      "outputs": [],
      "source": [
        "!cat configs/swift_yolo/swift_yolo_tiny_1xb16_300e_coco.py"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Y9sSOwrZdaLo"
      },
      "source": [
        "### Train the model using default config\n",
        "\n",
        "Then, train YOLOv5-Tiny model using default config, note here we need to override the config by 2 parameters:\n",
        "\n",
        "- `data_root` - the datasets path, which located at path `datasets/digital_meter`\n",
        "\n",
        "- `epochs`- the train epochs, we use `50` to reduce the training time\n",
        "\n",
        "- `num_classes` - the classes number of datasets, we use `11` here ['0', '1', '2', '3', '4', '5', '6', '7', '8', '9', 'N']\n",
        "\n",
        "For more training details, please refer to [SSCMA Documentation - Train YOLOv5-Tiny Model](https://seeed-studio.github.io/SSCMA/tutorials/training/yolov5)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "zUgKXe2s2wto"
      },
      "outputs": [],
      "source": [
        "!python tools/train.py \\\n",
        "  configs/swift_yolo/swift_yolo_tiny_1xb16_300e_coco.py \\\n",
        "  --cfg-options \\\n",
        "    epochs=10 \\\n",
        "    num_classes=11 \\\n",
        "    data_root='datasets/digital_meter/'"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tGl9VBZavSmM"
      },
      "source": [
        "The output directory in the training process would be look like this:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Ff8MtMaSu8Ij"
      },
      "outputs": [],
      "source": [
        "!ls -alh work_dirs/swift_yolo_tiny_1xb16_300e_coco.py"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "m4Y6DM16vgLF"
      },
      "source": [
        "We could inspect the latest model weight path by this command:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "RMf9sHP2vNzW"
      },
      "outputs": [],
      "source": [
        "!echo \"Latest model weights path: $(cat work_dirs/swift_yolo_tiny_1xb16_300e_coco.py/last_checkpoint)\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "G7KqGi2OxnaM"
      },
      "source": [
        "### Validate the Model\n",
        "\n",
        "We recommend you to validate the model with the latest weight after training using the `tool/test.py` script."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "v5SckZpKx21c"
      },
      "outputs": [],
      "source": [
        "!python tools/inference.py \\\n",
        "    configs/swift_yolo/swift_yolo_tiny_1xb16_300e_coco.py \\\n",
        "    \"$(cat work_dirs/swift_yolo_tiny_1xb16_300e_coco.py/last_checkpoint)\" \\\n",
        "    --dump work_dirs/swift_yolo_tiny_1xb16_300e_coco.py/last_checkpoint.pkl \\\n",
        "    --cfg-options \\\n",
        "        data_root='datasets/digital_meter/'\\\n",
        "        num_classes=11"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5np32KYq8JL4"
      },
      "source": [
        "## Optimize YOLOv5-Tiny Model and Export"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QF9ObhTrjwI2"
      },
      "source": [
        "### Export YOLOv5-Tiny PyTorch model to TFLite model\n",
        "\n",
        "Here we're using `tools/torch2tflite.py` to convert and export the latest weights to a TFLite model at a INT8 precision for inference."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "a50cBz7P8TP-"
      },
      "outputs": [],
      "source": [
        "!python tools/export.py \\\n",
        "  configs/swift_yolo/swift_yolo_tiny_1xb16_300e_coco.py \\\n",
        "  $(cat work_dirs/swift_yolo_tiny_1xb16_300e_coco.py/last_checkpoint) \\\n",
        "  --cfg-options \\\n",
        "    data_root='datasets/digital_meter/' \\\n",
        "    num_classes=11"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "akUdGWE-1Z-8"
      },
      "source": [
        "## Deploy to SenseCAP A1101\n",
        "\n",
        "This example is a tutorial for deploying the models from [SSCMA](https://github.com/Seeed-Studio/SSCMA/) to Grove - Vision AI module, based on the [Synopsys GUN Toolchain](https://github.com/foss-for-synopsys-dwc-arc-processors/toolchain) and [Tensorflow Lite Micro](https://github.com/tensorflow/tflite-micro) implementations.\n",
        "\n",
        "For more details, please refer to [SSCMA Documentation - Example - Grove - Deploy](https://sensecraftma.seeed.cc/SSCMA/examples/grove/deploy#compile-and-deploy)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1huhlfmt3np4"
      },
      "source": [
        "### Download and Setup Build Tool Chain"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "7hIBg6T73Ro3"
      },
      "outputs": [],
      "source": [
        "!wget https://github.com/foss-for-synopsys-dwc-arc-processors/toolchain/releases/download/arc-2020.09-release/arc_gnu_2020.09_prebuilt_elf32_le_linux_install.tar.gz -P ~/"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "HXrWQwVZ5I5O"
      },
      "outputs": [],
      "source": [
        "!tar -zxf ~/arc_gnu_2020.09_prebuilt_elf32_le_linux_install.tar.gz --directory ~/"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "WDBzyjFL7RHN"
      },
      "outputs": [],
      "source": [
        "SYSTEM_PATH = %env PATH\n",
        "SYSTEM_PATH = SYSTEM_PATH.replace('\"', '')\n",
        "SYSTEM_HOME = %env HOME\n",
        "SYSTEM_HOME = SYSTEM_HOME.replace('\"', '')\n",
        "SYSTEM_PATH = f'{SYSTEM_HOME}/arc_gnu_2020.09_prebuilt_elf32_le_linux_install/bin:{SYSTEM_PATH}'\n",
        "%set_env PATH={SYSTEM_PATH}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZMSi7Ie--O1h"
      },
      "source": [
        "### Clone and Setup SenseCAP A1101 SDK"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "vD6cIBo9-WPA"
      },
      "outputs": [],
      "source": [
        "!git clone https://github.com/Seeed-Studio/sscma-example-vision-ai example/grove"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "SZ0UyC7HA1zH"
      },
      "outputs": [],
      "source": [
        "!python -m pip install numpy requests colorama serial pyserial"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IGHvs_2n--gP"
      },
      "source": [
        "### Build Firmware"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "18TRCN-z-r3_"
      },
      "outputs": [],
      "source": [
        "!cd example/grove && \\\n",
        "  make HW=sensecap_vision_ai APP=digital_meter && make flash"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6ISXUUG2_uKY"
      },
      "source": [
        "### Convert Firmware Image to UF2"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "OJ5lwvPk_zhh"
      },
      "outputs": [],
      "source": [
        "!cd example/grove && \\\n",
        "  python tools/ufconv/uf2conv.py \\\n",
        "    -t 0 \\\n",
        "    -c tools/image_gen_cstm/output/output.img \\\n",
        "    -o firmware.uf2"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "i3effVeWCAhS"
      },
      "outputs": [],
      "source": [
        "!echo \"The UF2 firmware is located at: $(pwd)/example/grove/firmware.uf2\""
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "TFC2hSiXCfM3"
      },
      "outputs": [],
      "source": [
        "from IPython.display import FileLink\n",
        "\n",
        "FileLink('example/grove/firmware.uf2')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2zrBJSg0FST3"
      },
      "source": [
        "### Convert Trained Model (TFLite) to UF2"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "P4-YcaURF6K9"
      },
      "outputs": [],
      "source": [
        "!cd example/grove && \\\n",
        "  python tools/ufconv/uf2conv.py \\\n",
        "    -t 18 \\\n",
        "    -c \"$(cat ../../work_dirs/swift_yolo_tiny_1xb16_300e_coco.py/last_checkpoint | sed -e 's/.pth/_int8.tflite/g')\" \\\n",
        "    -o model.uf2"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "lMqfOQteIEL5"
      },
      "outputs": [],
      "source": [
        "!echo \"The UF2 model is located at: $(pwd)/example/grove/model.uf2\""
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "B-___RClH92g"
      },
      "outputs": [],
      "source": [
        "FileLink('example/grove/model.uf2')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_4cVSi76EsFg"
      },
      "source": [
        "Please download the UF2 Firmware and UF2 Model and flash them into a SenseCAP A1101 module. Finally, you can see the real-time meter reading results as shown in the figure below.\n",
        "\n",
        "![Meter Reader](https://seeed-studio.github.io/SSCMA/static/grove/images/digital_meter.gif)\n",
        "\n",
        "For more information on **how to use these UF2 images**, please refer to [SSCMA Documentations - Deploy - Grove - Deployment Routines](https://seeed-studio.github.io/sscma/examples/grove/deploy#deployment-routines)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qasiYT2qJWPm"
      },
      "source": [
        "## Thanks for Trying Out SSCMA 🎉\n",
        "\n",
        "Congratulations, you have completed this tutorial. If you are interested in more application scenarios or our projects, please feel free to give [SSCMA](https://github.com/Seeed-Studio/SSCMA) a star ✨ on GitHub.\n",
        "\n",
        "If you have any questions about this tutorial, please also feel free to [submit an issue](https://github.com/Seeed-Studio/SSCMA/issues)."
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "gpuType": "T4",
      "provenance": []
    },
    "gpuClass": "standard",
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.8.16"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}
